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(54) Digital video decoder for decoding digital high definition and/or digital standard definition 
television signals 



(57) Methods and apparatus for reducing the com- 
plexity of decoder circuitry and video decoder memory 
requirements are disclosed. The discribed video decod- 
ers are capable of decoding HDTV arxl/or SDTV pic- 
tures. The described video decoder may be used as part 
of a picture-in-picture decoder circuit for providing pic- 



ture-in-picture capability without providing multiple full 
resolution video decoders. The reduction in decoder cir- 
cuit complexity is achieved through the use of a plurality 
of data reduction techn'ques including the use of a pre- 
parser, downsanpling. and truncating pixel values. 
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Description 

Field of the Invention 

The present invention is directed to video decoders s 
and. more particularly, to methods and apparatus for 
implementing video decoders that are capable of decod- 
ing high definition television ("HDTV") signals and/or 
standard definition television f SDTV") signals. 

10 

Background of the Invention 

The use of digital, as opposed to analog signals, for 
television broadcasts and the transmission of other types 
of video arxi audio signals has been proposed as a way is 
of allowing improved picture quality and more efficient 
use of spectral bandwidth over that currently possible 
using analog NTSC television signals. 

The International Starxiards Organization has set a 
standard for video data compression for generating a 20 
compressed digital data stream that is expected to be 
used for digital television. This standard is referred to as 
the ISO MPEG (International Standards Organization - 
Moving Picture Exerts Group) ("MPEG") standard. In 
accordance with the MPEG standard, video data is 25 
encoded using discrete cosine transform encoding and 
is arranged into variable length encoded data packets for 
transmission. 

One version of the MPEG standard. MPEG-2. is 
desCTibed in the International Standards Organization - 30 
Moving Picture Experts Group, Drafts of Recommenda- 
tion H.262. ISO/IEC 13818-1 and 13818-2 titled "Infor- 
mation Technology - Generic Coding Of Moving Pictures 
and Associated Audio" (hereinafter Ihe November 1993 
ISO-MPEG Committee draft") hereby expressly incorpo- 35 
rated by reference. Any references made in this patent 
application to MPEG video data is to be understood to 
refer to video data that complies with MPEG-2 standards 
as defined in the November 1993 ISO-MPEG Committee 
drafts. 40 

MPEG video data may be used to support either 
high definition television ("HDTV"), wherein the video 
frames are of higher resolution than those used in 
present NTSC signals, or what will be referred to as 
standard definition television ("SDTV"), e.g., television 4S 
which has approximately the same resolution per frame 
as the existing analog NTSC standard. Because HDTV, 
which includes the proposed United States Advanced 
Television Standard ("US ATV"), provides higher resolu- 
tion than SDTV, more data is required to represent a so 
HDTV frame than is required to represent a SDTV frame. 
Accordingly, it is possil^e to transmit multiple SDTV sig- 
nals in the same bandwidth required to support a single 
HDTV signal. 

MPEG Main Profile at Main Level ("MP@ML") spec- 55 
if ies various requirements for an MPEG compliant stand- 
ard definition television signal and associated decoding 
equipment. MP@ML allows pictures as large as 
720x576 pels for a total of 414.720 pels per picture. The 



proposed standard for US ATV allows for pictures as 
large as 1920 X 1080 pels for a total of 2.073.600 pels 
per picture. 

Because of the relatively large amount of data 
required to represent each frame of a HDTV picture. 
HDTV decoders must support much higher data rates 
than SDTV decoders. The additional memory required 
by a HDTV decoder, as compared to a standard SDTV 
decoder, and the increased complexity associated with 
the inverse DCT circuit arxJ other components of a HDTV 
decoder can make a HDTV decoder considerably more 
expensive than an SDTV decoder. 

In fact, the cost of memory alone may make a HDTV 
set incorporating a HDTV decoder prohibitively expen- 
sive for some consumers. It is expected that a fully MPEG 
compliant video decoder for HDTV will require a mini- 
mum of 10 MB of RAM for frame storage with a practical 
HDTV decoder probably requiring about 16 MB of rela- 
tively expensive Synchronous DRAM. 

Accordingly, there is a need for a method and appa- 
ratus that permits: (1) a simplification of the corrplexity 
of the circuitry required to implement a HDTV decoder, 
(2) a reduction in the amount of memory required to 
implement a HDTV decoder circuit, and (3) a single 
decoder that is capable of decoding both SDTV and 
HDTV signals. Furthermore, it is desirable that the cost 
of such a decoder be low enough that it is in a range that 
would be acceptable to most consumers, e.g.. approxi- 
mately the cost of a SDTV decoder. 

V^hile various proposals call for transmitting only US 
ATV signals it has also been suggested that some digital 
SDTV signals be broadcast. Various combinations of 
broadcasting HDTV and SDTV signals are possible with 
multiple SDTV shows being broadcast during a particu- 
lar time of the day and a single HDTV broadcast being 
transmitted in the same bandwidth used for the SDTV 
signals at a different time of the day. 

Thus, in order to remain compatible with both HDTV 
and SDTV broadcasts, there would be value in a televi- 
sion receiver include a video decoder capable of decod- 
ing both HDTV and SDTV signals and furthermore, that 
such a video decoder be capable of being implemented 
at a relatively k)w cost. 

In addition, there is a need for a method and appa- 
ratus for implementing picture- in-picture capability in a 
digital television without incurring the cost of multiple full 
resolution decoders. In known analog pictu re-in-pi cture 
systems, during picture-in-picture operation, and full res- 
olution decoder is normally used to decode the TV signal 
used to produce a main picture and a second full reso- 
lution decoder is used to decode the television signal 
which is used to provide the second picture displayed 
within a small area of the main picture. 

Summary of the Present Invention 

The present invention is directed to video decoders 
and. more particularly, to methods and apparatus for 
implementing video decoders that are capable of decod- 
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ing high definition television ("HDTV") signals and/or 
standard definition compressed video signals. 

In particular the present invention is directed to a plu- 
rality of methods and apparatus for reducing the com- 
plexity of digital video decoder circuitry and for reducing 5 
video decoder memory requirements. 

A HDTV decoder capable of decoding HDTV signals 
at less than full HDTV resolution, e.g.. at approximately 
SDTV resolution and/or decoding SDTV signals is 
desaibed. It is expected that as a result of the various io 
data reduction features and circuit simplifications of the 
present invention a combined HDTV/SDTV decoder that 
outputs pictures at approximately SDTV resolution can 
be built at approximately 130% of the cost of a digital 
SDTV decoder. is 

In accordance with one embodiment of the present 
invention the described reduced resolution HDTV video 
decoders are used as part of a picture-in-picture decoder 
circuit for providing picture- in-pi dure capability without 
the use of muhiple full resolution video decoders. 20 

The decoder of the present invention achieves sub- 
stantial reductions in circuit complexity and memory 
requirements through the use of a plurality of data reduc- 
tion techniques including the use of a preparser, down- 
sampling, and truncating bytes of data representing pixel 25 
values so that the pixel values are represented using only 
5, 6 or 7 bits as opposed to. e.g., 8 bits per DCT value. 

In accordarrce with one embodiment of the present 
invention, the preparser is used to discard data to permit 
a smaller channel buffer than would otherwise be 
required to be used. In addition, the preparser discards 
run-iength/amplitude symbols for each macroblock when 
the number of run-length/amplitude symbols exceeds a 
maximum preselected number per macroblock. In one 
embodiment the preparser also limits the number of bits 
output per unit time period, e.g.. by limiting the number 
of bits per macroblock. such that a predetermined data 
processing rate which is determined by the combined 
capability of the coded data buffer and syntax parser and 
variable length decoder circuit is not exceeded. In this 
manner, the present invention permits a simpler real- 
time syntax parser and variable length decoder circuit to 
be used than would be possible without such preparsing. 

In accordance with the present invention, an inverse 
quantizer circuit and an inverse discrete cosine trans- 
form circuit are designed to process only a preselected 
set of DCT coefficients corresponding to each macrob- 
lock and to set the rest of the coefficients, e.g.. the DCT 
coefficients removed by the preparser, to zero. In this 
manner the computational requirements of the inverse 
quantizer circuit and inverse discrete cosine transform 
circuit are substantially reduced allowing for simpler cir- 
cuit implementation. 

In addition to the use of the preparser, the use of a 
downsampler arxl/or a least signif icarrt bit truncation ar- 
cuit provide for further reductions in the amount of data 
used to represent each video picture thereby resultir^g in 
a significant reduction in frame buffer memory require- 
ments. 



In order to compensate for the effect of data reduc- 
tion, including downsampling and data elimination, the 
decoder of the present invention may include low pass 
filters. 

The atx)ve described features and embodiments of 
the present invention along with numerous other features 
arxj embodiments are described below. 

Brief Description of the Drawings 

Figure 1 illustrates a video decoder in accordance 
with one exemplary embodiment of the present inven- 
tion. 

Figure 2A illustrates a video decoder in accordance 
with a secorxJ exemplary embodiment of the present 
invention. 

Figure 2B illustrates a video decoder circuit capable 
of decoding t>oth HDTV and SDTV video signals in 
accordance with the present invention- 
Figures 3A through 3E illustrate 8x8 blocks of DCT 
coefficients correspondir>g to a HDTV macroblock in 
accordance with different embodiments of the pres6nt 
invention. 

Figure 4 illustrates an exemplary circuit for decoding 
video images to support picture-in-picture television 
operation in accordance with the present invention. 

Detailed Description 

The present invention is directed to video decoders 
and, more particularly, to a low cost video decoder that 
is capable of decoding high definition and/or standard 
definition digital video signals. In accordance with the 
present invention decoded HD pictures will be displayed 
with less than full HD resolution, e.g., at approximately 
SD resolution, while SD pictures will be displayed at full 
SD resolution. The decoder of the present invention can 
be implemented as a low cost HDTV decoder or as a 
joint HD/SD television decoder. While the detailed 
description of the present invention is directed primarily 
to exenplary embodiments of HDTV, SDTV and com- 
bined HDTV/SDTV decoders, it is to be understood that 
the numerous features of the present irtvention nrtay be 
used to provide low cost reduced resolution digital video 
decoders suitable for decoding digital pictures in general , 
and that this disclosure is therefore not limited solely to 
SDTV and HDTV decoders. Furthermore, while the 
exenplary emlxxJiments are described as f^PEG com- 
patible decoders it should be noted that the decoders of 
the present invention can be implemented to decode a 
plurality of MPEG like video data streams that use dis- 
crete cosine transform data encoding. 

In the case of television sets with relatively small 
screens, e.g., less than 20 inch screens, the difference 
in perceived resolution and picture quality between a 
HDTV signal and an SDTV signal will be barely noticea- 
ble. Accordingly, low cost HDTV decoders in accordance 
with the present invention which provide less than full 
HDTV picture resolution and quality are particularly well 
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suited for such applications because of their cost advan- 
tages. The decoder of the present invention is also well 
suited for providing picture-in-picture capability because 
of the small display area of the second picture being dis- 
played and the relatively low cost of the decoder of the 
present invention as compared to the cost of a full HDTV 
decoder. 

The method of the present invention for decoding 
HD and SD pictures includes the steps of reducing the 
resolution of received HD pictures prior to decoding by 
using, e.g., a preparser unit and/or adaptive field/frame 
downsampling to reduce the complexity of later process- 
ing stages of the decoder In addition, by using an inverse 
discrete cosine transform ("IDCT") circuit designed to 
maximize the amount of common circuitry used for 
processing both reduced resolution HD and SD pictures 
the cost of implementing a combined HD/SD decoder cir- 
cuit is reduced. 

Referring now to Figure 2. there is illustrated a video 
decoder generally indicated by the reference numeral 
100, inplemented in accordance with one embodiment 
of the present invention. The illustrated decoder 100 is 
capable of decoding HD and/or SD television signals, 
e.g.. MPEG compliant television signals. When used as 
a HD decoder, the decoder circuit 100 can produce 
images from a HDTV signal that are comparable in qual- 
ity or slightly better than SD television images. 

The SD/HD video decoder circuit 100 conrprises a 
preparser 112, a syntax parser and variable length 
decoder circuit 120, an inverse quantizer 122. an inverse 
DCT circuit 124, a dovm sampler 126. a summer 128, a 
switch 129, a post processor 136, a motion compensated 
prediction ("MCP") circuit 130, and a memory circuit 114 
including a coded data buffer and a frame buffer 118. 

Methods of reducing the overall cost of a HDTV 
decoder will now be described with reference to the 
embodiment of Fig. 1 . 

Generally, the single most expensive element of a 
video decoder, in terms of cost, is the frame mennory 118 
which may comprise, e.g.. 16 MB of synchronous DRAM 
in a HD decoder. The coded data buffer 116. which is 
used for the temporary storage of the compressed bit- 
stream represents a smaller but not insignificant cost. A 
fully MPEG compliant HDTV decoder is expected to 
require at least 1 MB of RAM for use as a coded data 
buffer. 

Other elements of a decoder which add significantly 
to the cost of the decoder are the inverse discrete cosine 
transform circuit ("IDCT") 124 and the irrverse quantizer 
("IQ") circuit 122. The IDCT circuit 124 of a HDTV 
decoder is required to perform a large number of arith- 
metic computations at a high rate and therefore is likely 
to represent a significant portion of a decoder's circuitry. 
The lO circuit 122 performs a smaller number of compu- 
tations than the IDCT circuit 124 but because of the high 
speed and complexity of the computations, the cost of 
the IQ circuit 122 may also be a significant component 
of a HDTV decoder s overall cost- 



In addition to the IDCT circuit 124 and the IQ circuit 
122 motion compensation arx* interpolation logic may 
require a significant amount of logic gates, e.g.. the 
motion compensation and interpolation logic may require 
5 more logic gates than the IQ circuit to implement but less 
than the IDCT circuit. The syntax parser and variable 
length decoder circuit 120 may also require a significant 
amount of logic gates to implement. 

Because the cost and complexity of a HDTV 
w decoder is largely a function of the requirement that it 
process large amounts of data on a real time basis, it is 
possible to reduce the complexity and thus the cost of a 
HDTV compatible decoder by reducing the amount of 
data that needs to be processed. While using only a 
15 small portion of the video data received in an HDTV sig- 
nal will result in reduced resolution and picture quality, 
by carefully selecting which HDTV data to process and 
the method by which it is processed, video image quality 
comparable to or better than SD television signals can 
20 be achieved. As will be discussed below, the preparser 
112 serves to dynamically limit the amount of video data 
supplied to the remaining elements of the decoder circuit 
100 including the syntax parser arxJ VLD circuit 120 
thereby reducing the amount of data that must be proc- 
25 essed by the subsequent circuit elements on a real time 
basis and the required complexity of those circuit ele- 
ments. An additional benef rt of the use of the preparser 
1 12 is that it permits for the use of a smaller coded data 
buffer 1 16 than would otherwise be required. 
30 As illustrated in Fig. 2. the preparser 112 has afirst 
input for receiving the variable length encoded bitstream 
representing, e.g.. a HDTV signal output by a transport 
demultiplexer/decoder circuit of a receiver, and a second 
input for receiving data rate control information. The out- 
35 put of the preparser 1 12 is coupled to the input of the 
coded data buffer 1 1 6 which has a rate control output 
coupled to the second input of the preparser 1 1 2. 

The preparser 112 operates to reduce the amount 
of HDTV data supplied to the coded data buffer. It does 
40 this by selectively discarding some of the received HDTV 
data. By using the preparser 1 12 to parse and discard 
data prior to variable length decoding it is possible to 
operate the preparser 1 12 at a data rate that is a function 
of the data rate of the received bitstream. This rate will 
45 be considerably lower than the data rate that the corre- 
sponding circuitry would have to be capable of support- 
ing if it were located after the variable length decoder 
circuit 120. By preparsing the data prior to storing in the 
coded data buffer 116 and syntax parsing and variable 
50 length decoding, the need to support the vwjrst case 
processing requirements that might otherwise be 
encountered by the SP and VLD circuit 120 is avoided. 
Since the maximum data rate, e.g., the worst case data 
rate, that the SP and VLD circuit 120 would normally 
55 have to be designed to support can be as much as 60 
times higher than the average data rate, a substantial 
reduction in the complexity of the SP and VLD circuit 120 
can be achieved by using the preparser 112 to insure 
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that support tor what would normally be the worst case 
variable length decoding scenario need not be provided. 

The preparser 112 parses the incoming bitstream. 
without performing a complete variable length decode 
operation, to identity data corresponding to different s 
types of video frames, such as bi-directionally coded ("B- 
") frames, predictively coded video frames ("P-") frames 
and intra-coded ("I-") frames, and to identity MPEG cod- 
ing elements such as macroblocks including motion vec- 
tors and OCT coefficients associated with each of the to 
identified frames in the received data stream. 

The preparser 112 performs several data reduction 
operations on a HDTV bitstream as required to insure 
that the processing capacity of the SP and VLD circuit 
120 and throughout capability of the coded data buffer is 
1 16 are not exceeded. Thus, the preparser 112 operates 
to insure that the preparser output data rate stays within 
a real time decoding envelope established by the decod- 
ing limits of the other components of the decoder 100. 
By reducing the data rate as required, the preparser 112 20 
also serves to reduce the coded data buffer size require- 
ments because the coded data buffer 116 need only be 
capable of storing a reduced amount of data. 

In one embodiment, to insure that the preparser out- 
put data rate stays within the preselected limits of the 25 
decoding envelope, the preparser 112 limits the number 
of DCT coefficients used to represent a macroblock to a 
preselected number and also limits the number of bits 
per maaoblock by removing the highest order DCT coef- 
ficients associated with a received macroblock until the 30 
desired number of bits per macroblock required to pre- 
vent the coded data buffer capacity and decoding limits 
from being exceeded is achieved. Both the number of 
DCT coefficients per macroblock and the number of bits 
per macroblock can be preselected to approximately 3S 
achieve the desired data rate or can be dynamically 
selected by the preparser 112. 

Thus, in one embodiment, to limit its output data rate 
to a level required to prevent the coded data buffer 116 
from overt towing, the preparser 112 limits the number of 40 
DCT coefficients which are used to represent each mac- 
roblock by discarding the higher frequerrcy DCT coeffi- 
cients which exceed a preselected number of DCT 
coefficients. In such an embodiment, removal of AC DCT 
coefficients may be performed preferentially on B- 45 
frames, then P- frames and then I- frames, with a pref- 
erence for retaining data in earlier P- frames as opposed 
to later P- frames as measured from the most recently 
received I- frame, as required to achieve the desired 
reduction in the preparser output data rate. so 

In DCT based video compression systems the DCT 
coefficients are naturally ordered in two dimensions 
according to vertical and horizontal frequency bins. In 
typical systems the coefficients are assigned a one- 
dimensional ordering to allow for sequential transmis- ss 
sion. IVIPEG allows for two different orderings. generally 
called zig-zag scan and alternate scan. 

It may be the case that the subset of DCT coeffi- 
cients which are preserved, i.e.. the DC DCT coefficients 



and lower frequency AC DCT coefficients not discarded 
by the preparser 112 may not be consistent with any lin- 
ear range of coefficients in either of the MPEG scanning 
types. In such a case, it is difficult to limit the DCT coef- 
ficients to the desired set by discarding all of the coded 
coefficients of a macrotDlock after a certain point in the 
data stream. 

In accordance with the present invention, two differ- 
ent approaches are used to insure that the preparser 1 1 2 
discards the preselected DCT coefficients. In accord- 
ance with a first embodiment, the preparser 112 chooses 
the coefficient index in scan order after which to discard 
coefficients. In this embodiment the retained set of DCT 
coefficients will initially include all coefficients which do 
belong to the desired set as well as some coefficients 
which do not belong to the desired set. The preparser 
112 then sends only the DCT coefficients which belong 
to the desired set to the coded data buffer 116 to be 
passed to the SP and VLD circuit 120. 

In a second embodiment, the preparser 112 recodes 
the incoming bitstream. This is done by removing run 
length/amplitude codes which refer to undesired coeffi- 
cients, and by then recoding the later desired coefficients 
with suitably modified run lengths. 

By limiting the number of DCT coeff icients per mac- 
roblock. the preparser 112 effectively discards run- 
length/amplitude symbols which effectively cause zero 
DCT coefficient values thus allowing for a simpler IDCT 
circuit 124 to be used, in some embodiments, as 
opposed to the IDCT circuit that would be required to fully 
decode a HDTV bitstream. 

The discarding of run-length/amplitude symbols 
also allows for a simpler syntax parser and variable 
length decoder circuit 120 than would other wise be 
required because the use of the preparser 112 insures 
that no more than the preselected number of run- 
length/amplrtude pairs per macroblock will be output by 
the preparser 1 12 for further processing. 

Yet another benefit of the discarding of some DCT 
coefficients, i.e.. the effective setting of such coefficients 
to zero, is that the preparser 1 12 effects lowpass filtering 
on a macroblock basis thereby reducing or eliminating 
the need for additional filtering before a downsampling 
operation, which will be described below, is performed. 

In one particular embodiment, the preparser 112 
also operates to detect and eliminate data from the 
received bitstream which will have no impact on ultinnate 
picture quality. In one such embodiment, the preparser 
112 deletes from the data stream such MPEG data as 
bit slice stuffing and start code zero stuffing to thereby 
reduce the data rate without impacting picture quality. By 
eliminating such unnecessary data, the preparser output 
data rate and the size of the coded data buffer 1 16 can 
be reduced in this manner with no impact on picture qual- 
ity. 

In one embodiment of the present invention, the pre- 
parser 1 12 dynamically adjusts the amount of data dis- 
carding performed as a function of the amount of 
memory available in the coded data buffer 1 1 6 for storing 
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the received data. In order to facilitate such dynamic data 
discarding the coded data buffer 116 includes a data rate 
monitoring circuit which generates a signal supplied to 
the secorxJ input of the preparser 112 indicating how 
much data is stored at any given time in the coded data 5 
buffer 116. 

In such an embodiment, the preparser 112 main- 
tains a model of coded data buffer 1 16 occupancy and 
works to ensure that the buffer 116 will never overflow. 
As buffer occupancy increases as indicated by the signal 10 
received from the buffer 116. the preparser 112 becomes 
more severe in the amount of high frequency information 
that is discarded further reducing the number of DCT 
coefficients per macroblock from a preselected maxi- 
mum number to a lower number. is 

The preparser 112 can look to buffer occupancy 
characteristics of the received bitstream, e.g., the man- 
ner in which the video data was originally encoded for 
data transmission, and use this information as a factor 
In determining what information should be discarded to 20 
achieve the required reduction in the data rate. 

As one feature of the dynamic control of its output 
data rate and thus buffer control the preparser 112 com- 
putes and stores the number of bits used to code each 
row of macroblocks that conrprise a picture as the data 25 
is received. While this number may vary dramatically 
from row to row and picture to picture, since the decoder 
will take roughly the same amount of time to process 
each row or macroblocks regardless of the number of 
bits used to represent the row of macroblocks. the pre- 30 
parser can predict the rate of code buffer enrrptying. 
When this rate is expected to be relatively fast, e.g., 
because a large number of bits are used to represent the 
preceding rows of macroblocks, the preparser 112 can 
discard less data from the received data stream without 35 
concern of the coded data buffer 116 overflowing. How- 
ever, when relatively few bits are used to code the mac- 
roblocks representing the preceding rows of 
macroblocks. the coded data buffer 1 16 will empty on a 
comparatively stow basis requiring the preparser 1 12 to 40 
discard more data from the received data stream until 
the coded data buffer 116 becomes less occupied. 

In one embodimerrt, to minimize implementation 
costs, the preparser 112, which may also be referred to 
as a bitstream preprocessor, is designed to run at the 45 
incoming bit rate and is implemented using a bit serial 
architecture. Alternatively, the F>reparser 112 can parse 
an entire token at once, and at a rate that never needs 
to exceed the data rate of the received variable length 
encoded bitstream divided by the minimum length of a so 
symbol in the bitstream. 

As will be discussed below, when the decoder circuit 
100 is used as a joint reduced cost HDTV decoder and 
SDTV decoder, the data throughput is selected to permit 
the passing of a full rate SDTV bitstream. 55 

The coded data buffer 116. which is coupled to the 
preparser 112. in the at>ove described manner, is used 
to temporarily store the variable length encoded data 
output by the preparser 112 until the syntax parser and 



variable length decoding circuit 120 is ready to accept 
the video data for processing. 

The syntax parser and variable length decoder cir- 
cuit 120 has an input coupled to a data output of the 
coded data buffer 1 16 to receive therefrom the variable 
length encoded data temporarily stored therein. The syn- 
tax parser and variable length decoder ("SP and VLD") 
circuit 120 decodes the received data to a constant 
length bit stream. 

By using the preparser 1 12 in the above described 
manner, the computation requirements of the SP and 
VLD circuit 120 are substantially reduced as compared 
to the case where all the received data is syntax parsed 
and variable length decoded. The preparser 112 effec- 
tively limits the number of DCT coefficients which must 
be variable length decoded for any given macrot^lock by 
discarding coefficients exceeding the preselected maxi- 
mum number permitted per macroblock thus reducing 
the burden on the real time SP and VLD circuit 120 by 
putting a clear limit on the amount of data that will have 
to be processed for each macroblock. 

The output of the SP and VLD circuit 120 is coupled 
to the input of the inverse quantization ("IQ") circuit 122 
which, in turn, has an output coupled to the input of the 
IDCT circuit 124. 

Reduced complexity IQ and IDCT circuits 122 and 
124 are implemented in accordance with the present 
invention by intentionally setting preselected AC DCT 
coefficients corresponding to each macroblock to zero. 
The IQ circuit 122 can be designed to preform this func- 
tion. 

The IQ circuit 122 receives the data output by the 
SP and VLD circuit 120 and operates to generate DCT 
values of zero for preselected DCT coefficients regard- 
less of the input value of the preselected DCT coeffi- 
cients. In this manner, the IQ circuit need not actually 
process the preselected DCT coefficients received at its 
input but only the remaining DCT coefficients. 

In one exemplary embodiment, the IQ circuit 122 
sets all but the upper left 4X4 region of DCT coefficients 
of a set of 8X8 DCT coefficients representing a HDTV 
macroblock to zero. Fig. 3A illustrates the resulting effect 
on an 8x8 block of DCT coefficients representing a mac- 
roblock. In the illustration of Fig. 3, Xs are used to repre- 
sent DCT values that are actually processed by the IQ 
circuit 122. while as illustrated all the other DCT values 
are set to zero. 

Accordingly, in such an embodiment, the IQ circuit 
122 need only include circuity for performing inverse 
quantization on 16 of the possit>le 64 DCT coefficients 
corresponding to each macroblock and circuitry for out- 
putting zeros for the remaining DCT coefficient values. 
In this manner, the inverse quantization computation 
requirement is reduced by a factor of four. The conplexity 
of the IDCT circuit 124 is reduced in a similar manner 
since a preselected number of the DCT values will be set 
to zero by the IQ circuit 122. 

MPEG permits the use of either interlaced pictures 
or non-interlaced pictures. When encoding video data as 
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interlaced pictures, the DCT operation may have been 
pertormed on either a field or frame DCT basis. When 
interlaced video data is encoded on a frame DCT basis 
the high frequency coefficients represent temporal as 
well as spatial information. In such a case it may be desir- 5 
able to maintain high vertical frequencies to maximize 
the preservation of tenporal information. 

Accordingly, in one exemplary embodiment in which 
it is expected that some interlaced pictures encoded on 
a frame DCT basis will be received, the IQ circuit 1 20 is 10 
designed to force to zero all DCT coeff icients except the 
DCT coefficients corresponding to rows 0.1.6 and 7 and 
columns 0, 1 , 2. 3 of an 8x8 DCT coefficient two dimen- 
sional array representing a HDTV maaoWock. Such a 
case is illustrated in Fig. 3B where Xs are used to repre- 75 
sent DCT coefficient values that are processed by the IQ 
circuit 122 and zeros are used to represent the prese- 
lected DCT coefficients that are set of zero. 

It should be noted that it is possible to select DCT 
coefficients other than those illustrated in the embodi- 20 
ments of Figs. 3A and 3B to be set to zero depending on 
the desired implementation. 

The IDCT circuit 1 24 receives the data output by the 
inverse quantization circuit 122 and performs an inverse 
DCT operation on the received data. Because, in accord- 25 
ance with the present invention, preselected DCT coef- 
ficients were intentionally set to zero, the inverse DCT 
circuit need not include circuitry for processing those 
coefficients. Thus, as with the inverse quantization circuit 
122. the IDCT circuit 124 can be implemented using a 30 
relatively simple circuit. 

The output of the IDCT circuit 124 is coupled to the 
input of the downsampler 126. The downsampler 126 is 
used to downsample the data corresponding to each pic- 
ture prior to storage in the frame buffer 1 18. As a result ss 
of the downsampling operation, the amount of data 
required to represent a video frame is substantially 
reduced. For example if the downsampler 126 is imple- 
mented to remove half of the digital sanples used to rep- 
resent a picture, the amount of data that would have to 40 
be stored will be reduced by a factor of approximately 
two substantially reducing the amount of memory 
required to implement the frame buffer 118. 

Referring now briefly to Fig. 2A there is illustrated 
another embodiment of the decoder according to the 45 
present invention as indicated by the reference numeral 
200. Like numbered elements in Figs. 1 and 2A are the 
same or similar to each other and thus will not be 
described again in detail. In the decoder 200. a small 
buffer 202. e.g.. a buffer capable of storing the data cor- so 
resporxling to several lines of a video frame, in combi- 
nation with a low pass filter 204. is used to couple the 
output of the IDCT circuit 124 to the input of the down- 
sampler 1 26. The buffer 202 stores video data represent- 
ing several scan lines of a picture which is then low pass ss 
filtered by the low pass filter 204 prior to downsampling 
to reduce the blockiness in the image that might other- 
wise result from the downsampling operation. 
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Referring again to Fig. 1 it can be seen that the out- 
put of the downsampler 1 26 is coupled to a first input of 
the summer 128 and a first input of the switch 129. A 
second input of the summer 128 is coupled to the output 
of the MCP circuit 130 which supplies motion compen- 
sation information to the summer 128 to be combined 
with received P- and B- video frames. The video data 
output by the summer 128 represents a video frame that 
has been processed using motion compensation data to 
include the relevant video information from preceding 
and/or subsequent anchor frames. The output of the 
downsampler 126, on the other hand, represents either 
fully decoded picture data or decoded prediction residu- 
als in the cases of intraframe coded macroblocks and 
non-interframe coded macroblocks respectively. The 
switch 129 is controlled, as is known in the art. to selec- 
tively couple the output of the downsampler 126 to the 
input of the frame buffer 118 when the received video 
macroblock was not encoded using motion compen- 
sated prediction, e.g.. when the video data output by the 
downsampler was intra-f rame coded. However, when the 
video data output by the downsampler 1 26 was encoded 
using motion compensated prediction, e.g.. when it is B- 
frame or P- frame data, the output of the summer 1 28 is 
coupled by the switch 1 29 to the input of the frame mem- 
ory 118. 

The received downsampled. decompressed video 
frames are stored in the frame memory 118. The frame 
memory 118 has a first output coupled to an input of the 
MCP circuit 1 30 and a second output coupled to an input 
of the post processor circuit 1 36. In this manner the MCP 
circuit is supplied with downsampled video frames to be 
used as anchor frames for motion compensation pur- 
poses. 

Referring again briefly to Fig. 2A, it can be seen that 
the decoder 200 contains additional circuitry for reducing 
the amount of data required to be stored by the frame 
buffer 118. i.e., a least significant bit ("LSB") truncation 
circuit 208. The LSB circuit 208 couples the output of the 
switch 129 to the input of the frame buffer 118. MPEG 
assumes that 8 bits will used per pel when represent- 
ing a video frame. In accordance with the present inven- 
tion, in order to reduce frame buffer memory 
requirements, one or more least significant bits are trun- 
cated from each set of bits representing a pel. For. exam- 
ple, in one embodiment, the LSB truncation circuit 208 
truncates the data representing a pel from 8 bits to e.g., 
6 or 7 bits. When the data corresponding to each pel is 
truncated to 6 bits a data savings of approximately 25% 
results. 

The amount of truncation performed may be imple- 
mented as a function of the type of frame to which the 
data corresponds. Because reduction in the resolution 
in anchor frames will impact the quality of frames which 
use these frames for motion compensation purposes, it 
may be desirable to reduce the resolution of these 
frames less than frames, such as B- frames, which do 
not impact the quality of other frames. Accordingly, in one 
embodiment anchor frame data. e.g.. I- and P- video 
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Irame data, is truncated to 7 bits per pel while data which 
is not used for anchor frames, e.g. . B- frame data, is trun- 
cated to 5 bits per pel. 

VS^en LSB truncation is used to reduce frame buffer 
memory requirements, in order to maintain MPEG com- 5 
patibillty it is necessary to represent the stored data in 
an 8 bit per pel format. Accordingly, the outputs of the 
frame buffer 1 18 are coupled to least sign'rficant bit pad- 
ding circuits 206. 207 which pad the least significant bits 
of the data output by the frame buffer 1 18 with zeros to w 
insure that the data corresponding to each pel of a frame 
is represented by 8 bits. The output of the LSB padding 
circuit 206 is coupled to the input of the MCP circuit 
thereby providing the MCP circuit with downsampled 
video frame data. The output of the LSB padding circuit is 
207 is coupled to the input of an optional low pass filter 
209 which, in turn, is coupled to the post processing cir- 
cuit 136. The optional low pass fitter 209 reduces the 
effect of the quantization noise generated as a resuH of 
the truncation and padding process but also results in a 20 
slight reduction in the resolution of the video frame being 
supplied to the post processing circuit 136. 

The operation of the MCP circuit 130 illustrated in 
Figs. 1 and 2A. which is designed to operate using 
motion vectors generated in accordance with the MPEG 2s 
standard, will now be described. 

MPEG supports the use of motion compensation 
and, In particular. P- frames and B- frames each of which 
rely on another frame as an anchor frame. The motion 
vectors received as part of the video data stream sup- 30 
plied to the preparser 1 12 were calculated at encoding 
time for use with full resolution video frames as opposed 
to downsannpled video frames. 

In accordance with the present invention, the down- 
sampled frames supplied by the frame buffer 1 18 to the 3S 
MCP circuit 130 are upsampled, e.g., on-the-fly, interpo- 
lated and then downsampled prior to generating predic- 
tions based on the nrK>tion vectors. In this manner the 
motion vectors which were originally generated based 
on full resolution video frames are effectively applied to 40 
downsampled video frames. 

As illustrated in Fig. 1, the MCP circuit 130 com- 
prises an upsanrpter 131 which has an input coupled to 
the first output of the frame buffer 118. and an output 
coupled to the input of a half pel interpolation circuit 132. 45 
The output of the half pel Interpolation circuit 1 32 is cou- 
pled to the input of a motion conrpensated prediction 
module 1 35. The output of the motion compensated pre- 
diction module 135 is coupled to a downsampler 133 
which, in turn, has an output coupled to the input of the so 
average predictions circuit 134. The output of the aver- 
age predictions circuit 134. which serves as the output 
of the MCP circuit 1 30. is coupled to the second input of 
the summer 128. 

Various known upsampling and down sampling fil- 55 
ters may be used for the upsampler 131 and downsam- 
pler 133, respectively. However, for best results, 
simulations have shown that it is very important that the 
downsampler 133 use a downsampling process which is 



the exact inverse of the upsampling process since even 
small differences are made noticeable in inter-coded 
video frames after many generations of predictions. 

There are several ways the required upsampling and 
downsampling can be accomplished. For example, in 
one embodiment a bilinear upsampling filter with a 3x3 
kernel and a center coefficient = 1.0 is used as the 
upsampler 131 and a downsampler which does not per- 
form filtering is used as the downsampler 133. This 
selection of fitters allows for reasonably good upsam- 
pling with no additional blurring during downsampling. 
This filter arrangement works especially well when the 
residual image to be accumulated during predication has 
been lowpass filtered prior to accumulations. Such low 
pass filtering Is, as discussed atx)ve, achieved to some 
extent as a result of the preparser 112 removing higher 
order AC DCT coefficients from macroblocks which 
include more than the preselected maximum number of 
DCT coefficients per macroblock. 

In another emtMxJiment an upsampling filter with a 
kernel size of 2x2 pixels is used as the upsampler 131 
and a d6wnsarnpling fitter is used with the same kernel 
size as the upsampling filter and a transfer function such 
that the convolution of the upsampling filter transfer func- 
tion and the downsanpling filter transfer function pro- 
duces a 3x3 kernel whose center coefficient is 1 .0. 

While it is preferred that identical downsampling cir- 
cuits be used for the downsampler 126, 123 there is no 
requirement ttiat this in fact be the case. However, it is 
inrportant that the downsarrpler 1 33 be matched to the 
upsannpler 131. as discussed atxive, to provide good 
results. 

MPEG permits the portion of an image correspor»d- 
ing to a macroblock to be encoded either on an interlaced 
or non-interlaced manner. Macroblocks corresponding 
to non-interlaced pictures are always coded using a 
frame DCT format and with macrot^tocks corresponding 
to interlaced pictures being coded using either a field or 
frame DCT format. 

In accordance with the present invention, downsam- 
pling is performed separately on each individual macrob- 
lock. Downsampling of macroblocks corresponding to 
non-interlaced pictures is performed by the downsam- 
pler 126 on a frame basis. 

However, with regard to macrot^locks corresponding 
to interlaced pictures downsampling can be performed 
on either a field or frame basis for each macroblock. 

As discussed atxrve, the downsampler 1 26 performs 
downsampling on non-interlaced pictures on a frame 
basis. However, with regard to interlaced pictures it can 
be performed on either a f iekj or frame basis. Thus, in 
the case of maaolDlocks of interlaced pictures one is 
confronted with a decision whether the downsampling, 
performed by the downsamplers 126, 133, should be 
done on a field or on a frame basis. Field t>ased down- 
sampling preserves the greatest degree of temporal res- 
olution whereas frame based downsampling has the 
potential 0I preserving the greatest degree of spatial res- 
olution In one embodiment of the present invention pic- 
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tures are downsampled in a consistent manner either on 
a field or frame basis when interlaced pictures are 
received. Accordingly, in one embodiment both inter- 
laced and non-interlaced pictures are downsampled on 
a frame basis. While in another embodiment non-inter- 
laced pictures are downsampled on a frame basis while 
interlaced pictures are downsampled on a field basis. 

In another slightly more elaborate embodiment, the 
downsampler 126 performs downsampling on a inter- 
laced pictures on both a field arxJ frame basis with the 
particular method depending on the individual macrob- 
lock being processed. 

Field based downsanpling is most appropriate 
when there is not much high frequency content in the 
macroblock being downsampled and/or there is a great 
deal of motion associated with the macroblock. On the 
other hand, frame based downsampling is most appro- 
priate when there is significant high frequency content in 
the macroblock being processed and/or little motion 
associated with the maaoblock. Accordingly, in one 
embodiment, the downsampler 126 assesses the 
described characteristics for each macroWock of an 
interlaced picture and dynamically determines the most 
appropriate method for downsampling the individual 
macroblock. 

In yet another embodiment which supports down- 
sampling on both a field and frame basis but which 
requires relatively little circuitry for deciding which down 
sampling approach to use. the downsampler 126 merely 
follows the choice made by the encoder when it originally 
encoded the macroblock being downsampled since the 
criteria used for previously deciding whether to encode 
the macroblock on a field or frame DCT basis is similar 
to the criteria which should be used to decide whether 
to downsanrple on a field or frame basis. 

Accordingly, in one embodiment of the present 
invention the downsampler 126 downsamples macrob- 
locks corresponding to an interlaced picture on a frame 
basis if they were encoded on a frame DCT basis and 
downsamples them on a field basis if they were encoded 
on a field DCT basis. 

While the downsampling of frames prior to storage 
in the frame buffer 114 reduces frame buffer memory 
requirements, it has the disadvantage of requiring the 
use of the upsampter 131. half pel interpolation circuit 
1 32 and downsampler 1 33 in order to take full advantage 
of the motion vectors. Such circuitry slightly increases 
the processing requirements as compared to those of 
standard MCP circuits 15 since the anchor picture data 
must be upsampled prior to interpolation if half -pel res- 
olution is to be achieved. H should be noted that as an 
alternative to upsampling, interpolation, and then down- 
sampling, a less complicated and thus less costly way to 
perform motion compensation, with somewhat lower 
quality results, is to divide the value of each origirial 
motion vector by the integer decease in resolution ar>d 
take the average integer result of the division operation 
as the new motion vector value. 



While downsampling thus tends to increase MCP 
processing requirements slightly it has the advantage of 
reducing the complexity of the input/output circuitry to the 
MCP circuit because the data bandwidth into and out of 
5 the MCP circuit 130 is reduced as a result of the down- 
sampling by the same factor as the buffer frame memory 
requirements. 

In accordance with one embodiment of the present 
invention, in order to reduce the processing and band- 
w width requirements of the MCP circuit 1 30. and thus the 
complexity of the MCP circuit 130. bi-directional and 
dual-prime prediction is avoided. Thus eliminating the 
need for the average predictions circuit on the ability to 
process to anchor frames simultaneously. Once a picture 
IS has been upsampled by the upsampler 131 interpolation 
is preformed by the half pel interpolation circuit. The 
nnotion compensated prediction module then actually 
applies the motion vectors to the upsampled picture 
which is then downsampled by the downsampler 133. 
20 When the picture being processed is a bi-directionally 
coded picture, the upsampling interpolation, motion and 
prediction operations will be performing on two anchor 
franries which are then averaged together by the average 
predictions circuit to generate the single picture output 
25 by the MCP circuit 135. 

This is accomplished, in accordance with one 
embodiment of the present invention by using only a sin- 
gle reference or anchor picture for prediction purposes 
even if the macroblock type flag associated with a mac- 
30 roWock being processed indicates that it is a bi-direc- 
tional coded n^croblock. i.e., a macroblock that was 
coded based on the content of two different pictures. 

In such an embodiment when the bi-directionally 
coded macroblocks are received, the MCP circuit 130 
35 performs motion compensation using only one reference 
despite the fact that the macroblock type flag associated 
with the individual macroblock being processed in the 
data stream indicates that motion connpensation should 
be performed using two references. 
40 In an exenrplary embodiment, the upsampler circuit 
131 detects when a bi-directionally or dual coded mac- 
roblock is being processed, selects which of the two f>os- 
sible anchor pictures associated with the nnacroblock to 
use for predication purposes and then upsamples only 
45 the selected anchor picture. 

In one embodiment, the upsampler circuit 131 
selects which of the two possible anchor pictures to use 
for predication purposes by selecting the anchor picture 
that is temporally closet to the current picture and thus 
50 macroblock being processed. If the two anchor pictures 
are temporally equidistant, the average predictions cir- 
cuit 134 selects the earlier of the two possit>le anchor 
pictures to thereby select the one of the two anchor pic- 
tures which will have suffered the least degradation due 
55 to drift. 

Since errors do not propagate from B-pictures, 
because they are not used as anchors for subsequent 
pictures, a fairly large degree of distortion, i.e.. of the type 
caused by using only a single anchor picture for predic- 
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tion purposes, can be tolerated with regard to B-pictures. 
However, the use of a single anchor picture for prediction 
purposes with dual-prime coded sequences may result 
in somewhat less satisfactory results because prediction 
errors may propagate when dual-prime coded 
sequences are used. 

Referring now to Fig. 2B there is illustrated a video 
decoder circuit 300 according to the present invention 
capable of decoding HDTV signals at less than full HDTV 
resolution and SDTV signals at full SD resolution. As 
illustrated, the decoder circuit 300 is similar to the 
decoder circuit 100 but includes additional circuitry, i.e.. 
a HDTV/SDTV mode control circuit 310 for controlling 
the decoder circuit's mode of operation, a set of switches 
301. 302. 303 and a multiplexer 341 for bypassing the 
decoder's preparser 112. downsampler 126. upsampler 
131 . and downsampler 1 33 when the HDTV/SDTV mode 
control circuit 310 determines that an SDTV signal is 
being received. When the HDTV/SDTV mode control cir- 
cuit 310 determines that a HDTV signal is being received 
it controls switches 301 , 302. 303 and multiplexer 341 to 
connect the circuit elements in generally the same man- 
ner as illustrated in the embodiment of Fig. 1 so that the 
data reduction features of the decoder circuit 300 of Fig. 
28 are operative. 

The HDTV/SDTV mode control circuit 310 receives 
as its input the variable length encoded brtstream. From 
analyzing the data in the bitstream, e.g., header informa- 
tion indicating the resolution of the video frames being 
received, it determines if it is receiving HDTV data, i.e.. 
data of a higher resolution than it can decode without 
performing data reduction operations on the received 
data, or SDTV data, i.e. data which the decoder 300 can 
fully decode and store without having to perform any data 
reduction operations. 

The HDTV/SDTV mode control circuit 310 gener- 
ates a mode control signal which is supplied to a position 
control input of the switch 301 which is part of the pre- 
parser module 312, a position control input of the switch 
302 which is part of a downsarrpler module 326 and a 
position contrd input of the switch 303 and multiplexer 
341 which is part of a motion vector interpolation module 
333. As discussed above, the output signal of the 
HDTV/SDTV mode control circuit 310 operates to switch 
each of the modules 312, 326, and 333 between a first 
mode of operation, e.g., a SDTV mode of operation, to 
a second mode of operation, e.g.. a HDTV mode of oper- 
ation. In such a manner, the decoder 300 is controlled 
by the HDTV/SDTV mode control circuit to perform data 
reduction to reduce the resolution of the received frames 
during the second mode, i.e., the HDTV mode of opera- 
tion, but not during the first mode of operation, i.e. the 
SDTV mode of operation. 

While the frame buffer 118. coded channel buffer 
116. IQ circuit 122. IDCT circuit 124. MCP circuit 330 
and SP and VLD circuit 1 20 are generally the same in 
the decoder 300 as the decoder 100, various features of 
each of these decoder elements will be discussed as 
they relate to the particular embodiment illustrated in Fig 



2B. e.g.. when they are implemented as part of a decoder 
which is capable of decoding both a television signal of 
a first resolution and a second television which is of lower 
resolution than the first television signal, e.g.. HD and 
5 SD television signals. 

The preparser 1 12. in the emlxxiiment illustrated in 
Fig. 2B. is designed to reduce the data rate of a received 
HDTV signal to approximately that of a SDTV signal. 
Thus, the preparser 112 limits the number of run- 
10 length/annplitude syrrtools per coded macroblock and the 
number of bits per macrot)lock so that the SP and VLD 
circuit 120 can function at approximately the same rate 
when processing HDTV signals as when processing 
SDTV signals, e.g., at the MP@ML specified maximum 
15 data rate. 

This reduces the memory requirements for the 
coded data buffer 116 which serves as a channel buffer. 
Thus, the decoder 300 uses a channel buffer that meets 
the memory requirements for a SDTV decoder but which 
20 is smaller than that required for a fully MPEG compliant 
HDTV decoder. U.S. ATV specifies a channel buffer of 8 
Mb, whereas only 1 .5 Mb are required to implement a 
MP@ML channel buffer. Because the MP@>ML specifi- 
cation requires 830 Kbits more for a frame buffer than 
25 are required for decoding reduced resolution video pic- 
tures in accordance with one emtjodiment of the present 
invention, this 830 Kbits of the frame buffer which are not 
used for picture storage during HDTV mode operation 
can be used to supplement the memory of the channel 
30 buffer when operating in HDTV mode. Thus, in accord- 
ance with the present invention, the coded data buffer is 
implemented using approximately 2.33 Mbits which are 
also required to implement a full resolution SD decoder. 
It is the responsibility of the preparser 1 12 to insure 
35 that the coded channel buffer 116 does not overflow dur- 
ing HDTV mode operation. This is achieved by, e.g., the 
^ preparser 112 discarding the less useful DCT coeffi- 
cients in order to prevent the smaller than full size HDTV 
channel buffer 116 from overflowing. 
40 While, the SP and VLD unit 120 is designed to oper- 
ate at the same general data rate as a SDTV data rate, 
it is also designed to accomrrxxlate the larger range of 
parameter values allowable for HDTV coding, as com- 
pared to SDTV decoding, e.g., frame rate, picture hori- 
45 zontal size and picture vertical size. Thus, the SP and 
VLD circuit 120, because of the larger range of parame- 
ter values required to support both HDTV and SDTV 
decoding, is slightly more complicated then a standard 
SDTV syntax parser and variat^e length decoder circuit 
50 in this regard. 

As described above, the output of the syntax parser 
120 is coupled to the input of the IQ circuit 122 which, in 
turn, is coupled to the IDCT circuit 124. The IQarxJ IDCT 
circuits 122, 124 of the decoder 300 will now be 
55 described. 

MPEG MP@ML allows for pixel rates of up to 10 4 
million pixels per second. U.S. ATV allows for pixel rates 
of up to 62.2 million pixels per second. 
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In accordance with the present invention. IQ circuitry 
which meets MP@ML IQ requirements is used tor the IQ 
circuit 122 of the decoder 300. In such an embodiment, 
the IQ circuit 1 22 is designed to process all the DCT coef- 
ficients of a SDTV macroblock but only, e.g.. 10 or 11 5 
DCT coefficients of an HDTV macroblock. 

That is. the IQ circuit 122 is designed to set a prese- 
lected group of DCT coefficients of a HDTV macroblock 
to 0 while processing the remaining 10 or 1 1 DCT coef- 
ficients of each HDTV macroblock being processed. 10 

In one exemplary embodiment, for HDTV macrob- 
locks coded using field DCTs. the DCT coefficients of an 
8x8 macroblock are discarded by the IQ circuit 122 as 
illustrated in Fig. 3C where zeros are used to represent 
discarded DCT coefficients and X's are used to repre- 15 
sent DCT coefficients which are processed by the IQ cir- 
cuit 122. For HDTV macroWocks coded using frame 
DCTs, the DCT coefficients of an 8x8 macroblock are 
discarded by the IQ circuit 122 as illustrated in Fig. 3D. 

The IDCT circuit 124. like the IQ circuit 122 is 20 
designed to processes only a small percentage of the 
DCT coefficients associated with a HDTV macroblock, 
e.g., the same DCT coefficients processed by the IQ cir- 
cuit 122. 

An IDCT engine capable of performing fast one 2s 
dimensional 8 point IDCTs can be used to implement the 
2-D IDCT operation required by the IDCT circuit 124. A 
standard MP@ML IDCT engine can be used to compute 
about three 8 point IDCTs in the time allowed tor IDCT 
processing of one HDTV macroblock. Accordingly, in one 30 
embodiment of the present invention, only DCTs in the 
first three columns of an HDTV maaoblock are proc- 
essed by the IDCT circuit 124. e.g.. the DCT coefficients 
indicated by Xs in Figs. 3C and 3D. 

Accordingly, in one embodiment the IDCT circuit 1 24 35 
includes a MP@ML compliant IDCT engine for calculat- 
ing IDCTs of the first three columns of a HDTV macrob- 
lock and all the IDCTs of a SDTV macroblock Since, in 
accordance with the exemplary embodiment of the 
present invention, the DCT coefficients of all but the first 40 
three columns of a 8x8 HDTV macroblock are set to zero, 
there is no need to calculate the IDCTs for any coeffi- 
cients outside the first three columns. 

While the MP@»ML IDCT engine is used to calculate 
the column IDCTs, a IDCT engine for calculating the row 45 
IDCTs for a HDTV macroblock is also included in the 
IDCT circuit 124. However, in accordance with the 
present invention, this can be a relatively simple IDCT 
engine since 5 of the 8 row coefficients will always be 
zero by the preselected setting of such coefficierrts to so 
zero. Furthermore, because only the first two or three 
columns contain coefficient values that are to be used in 
the calculation of row DCTS only 2 or 3 points need to 
be calculated for each row transform. Note that in the 
embodiment which uses the 8x8 DCT arrangement illus- 55 
trated in Figs. 3C and 3D only 4 rows have to be trans- 
formed when no additional filtering is performed prior to 
downsarrpltng by the downsampler 126. 
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As an alternative method of implementing an IDCT 
circuit 124 which uses an MP@ML IDCT engine as part 
of HD decoder IDCT circuit, the low frequency rectangle 
of coefficients from several macroWocks may be com- 
bined into a single macroblock as illustrated in Fig. 3E, 
where "1" is used to represent low-frequency coefficients 
from a first macroblock, "2" is used to represent low-fre- 
quency coefficients from a second blocks, "3" is used to 
represent low-frequency coefficients from a third mac- 
roblock and "4" is used to represent low frequency DCT 
coefficients from a fourth macroblock. 

In accordance with this alternative embodiment, the 
IDCT circuit 124 arranges the DCT coefficients from four 
different HDTV macroWocks into a single 8x8 block of 
DCT coefficients in the described manner and a conven- 
tional 8x8 point IDCT engine is then used to process the 
composite block of DCT coefficients. After processing by 
the 8x8 point IDCT engine relatively simple linear oper- 
ations performed on the result of the IDCT operation are 
used to generate a good approximation of the IDCTs of 
the low frequency coefficients of each one of the four 
component HDTV nrecroblocks used to form the com- 
posite coefficient block. 

As discussed above, the video picture data output 
of the IDCT circuit 124 is downsampled when the 
decoder 300 is operating in HDTV mode prior to storing 
in the frame buffer 1 1 8 to thereby further reduce memory 
requirements. 

MP@ML allows for pictures as large as 720x576 
pels for a total of 414720 pels per picture while US ATV 
allows for pictures as large as 1920x1080 pels per pic- 
ture. Sequences received in the 1920x1080 pel picture 
format can be downsampled by a factor of 3 horizontally 
and a factor of 2 vertically to yield a maximum resolution 
of 640x540 pels for a total of 345.600 pels. In such an 
embodiment, the same amount of memory required to 
implement a full resolution SD decoder is satisfactory for 
storing a downsampled HDTV video picture. Thus, the 
memory provided for MP@ML would be adequate for the 
reduced resolution HD decoder as well, with an addi- 
tional 830 KBits available for other purposes, e.g.. for use 
as part of the coded data buffer 116 wrhen operating in 
HDTV mode as described above. 

By selecting decoder components that are designed 
in the above manner, it is possil^le to implement a com- 
bined low resolution HDTV decoder and full resolution 
SD decoder at a cost that is expected to be about 130% 
of the cost of a full resolution SDTV decoder. The 
decoder 300 may be used as part of a television set 
which is designed to decode and display both HDTV and 
SDTV signals at a relatively low cost as connpared to a 
system incorporating a full resolution HDTV decoder 

The decoders illustrated in any of Figs. 1 and 2A-2B 
may also be used to implement picture-in-picture capa- 
bility in a HDTV or SDTV set without incurring the cost 
of having to provide a full resolution decoder for the 
decoder used to display the picture shown inside the 
main picture of a picture-in-picture display. 
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For a MPEG compatible digital television set to oper- 
ate in a picture-in-picture mode, two MPEG decoders 
must be used . a first decoder to decode the full resolution 
picture to be displayed on a first portion of the screen 
and a second decoder to decode the reduced size pic- 5 
lure displayed on a second smaller portion of the screen. 
Referring now to Fig. 4, there is illustrated a picture-in- 
picture ("PIP") decoder 400 in accordance with the 
present invention. The PIP decoder 400 receives MPEG 
compliant denrodulated digital video information, e.g., w 
either an SD or HD TV signal, from a tuner/demodulator 
399 which is supplied to the input of a primary, e.g.. full 
resolution decoder 401 and to the input of one or more 
reduced resolution decoders, i.e., the first and second 
reduced resolution decoders 402, 403. 75 

The primary decoder 401 is responsible for decod- 
ing the main picture of a picture-in-picture image while 
the first and second decoders are responsible for gener- 
ating separate images which will be displayed in a small 
area of the main picture. A separate reduced resolution 20 
decoder 402 or 403 is used for each additional image 
that is to be displayed in addition to the main picture. 

The output of the primary decoder 401 and the 
reduced resolution decoders 402. 403 is coupled to the 
input of a picture-in-picture video processing circuit 2S 
which operates to combine the main picture with the 
reduced resolution pictures output by the reduced reso- 
lution decoders 402. 403 prior to the resulting combined 
picture being displayed. 

In one embodiment of the present invention the size 30 
of the reduced resolution pictures incorporated into the 
main picture is selected to be 1/4 x 1/4 the size of the 
normal picture. In such an embodiment, each MPEG 8x8 
pixel block need only be decoded to a size corresponding 
to a block of 2x2 pixels. 35 

The cost of the IDCT circuit 1 24 used in the reduced 
resolution decoders 402, 403 can be substantially 
reduced in accordance with the present invention by per- 
forming the IDCT operations on only 2x2 blocks as 
opposed to 8x8 blocks. This is achieved by, e.g.. retaining 40 
arxJ processing only the upper left 2x2 block of DCT coef- 
ficients of each 8x8 DCT coefficient block of a HDTV pic- 
ture with all the other DCT coefficients being set to zero. 
Accordingly, in such an embodiment, the IDCT circuit 
cost is reduced to approximately the cost of a circuit 45 
which can perform a 2x2 IDCT as opposed to an 8x8 
IDCT 

The IQ circuit 122 of the reduced resolution decod- 
ers 402. 403 can be simplified in a similar manner with 
the IQ circuit 122 operating only on a 2x2 block of DCT 50 
coefficients, i.e.. 4 coefficient values, as opposed to 64 
DCT coefficient values that form an 8x8 DCT coefficient 
block. 

In accordance with the present invention, the 
reduced resolution decoders 402. 403 store the low res- 55 
olution frames in their reduced size. Thus, by using 2x2 
DCT coefficieni blocks the size of the frame buffers can 
be about 1/I6th of the size that would be required if 8x8 



DCT coefficient blocks, i.e.. full resolution blocks, were 
stored. 

In addition to reducing frame buffer size the pre- 
parser 112 serves to reduce channel buffer requirements 
and the complexity of the SP and VLD circuit by, in 
accordance with the PIP embodiment, discarding ampfi- 
tude/runlength pairs corresponding to each macroblock 
that exceed the 4 amplitude/runlength pairs that are used 
by the reduced resolution PIP decoder 402 or 403. 

In a PIP embodiment, the MCP circuit 130 can per- 
form interpolation on the reference frames for macrob- 
locks whose motion vectors are not evenly divisible by 
four to achieve good prediction results. For a less expen- 
sive and somewhat lower quality implementation, motion 
vectors could be rounded to the nearest factor of 4 and 
applied without interpolation. 

While the above PIP decoder embodiment of the 
present invention has been described in the context of 
using 1/4x1/4 size pictures for the secondary pictures 
inserted into the primary picture, other size pictures 
could also be generated at reduced cost. e.g.. 1/2 x 1/2 
size pictures, using the decoder 100 described above. In 
such a case, the downsampling would have to be per- 
formed in a manner that would produce the desired 
reduction in size and data rate. 

It should also be noted that the arrangement illus- 
trated in Rg. 4 can be used to provide picture-in-picture 
capability in a SDTV receiver as well as a HDTV receiver. 
Furthermore, the illustrated PIP decoder arrangement is 
in no way limited to a specific degree of resolution with 
regard to the primary decoder 401 and only requires that 
the secondary decoder(s) 402. 403 be implemented as 
reduced resolution decoders as compared to the resolu- 
tion supported by the primary decoder. 

Claims 

1. A decoder circuit for decoding digital video data 
including pictures at a resolution less than the full 
resolution of the pictures, each picture including a 
plurality of macroblocks, each nrracroblock including 
a plurality of DCT coefficients, the decoder circuit 
comprising: 

preparser means for receiving the digKal 
video data and for removing from each macroblock 
DCT coefficients that exceed a preselected maxi- 
nuim number of DCT coefficients per macroblock; 
and 

a channel buffer coupled to the preparser 
means for temporarily storing the video data output 
by the preparser means. 

2. The decoder of claim 1 . 

wherein each DCT coefficient is represented 
by a number of bits and wherein each macroblock is 
represented by the total number of bits representing 
the DCT coefficients included in the macroblock. 
and 

wherein the preparser means operates to 
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remove DCT coefficients from each individual mac- 
roblock until the number of bits used to represent the 
individual macroWock does not exceed a predeter- 
mined maximum number of bits. 

5 

3. The decoder of claim 1. wherein the channel buffer 
is further coupled to the preparser by a feedback line 
and wherein the channel buffer signals the preparser 
to control the output data rate of the preparser as a 
function of the amount of data stored in the channel io 
buffer. 

4. The decoder of claim 1 . further comprising 

a syntax parser and variable length decoder 
circuit coupled to the channel buffer; and 75 

an inverse quantization circuit coupled to the 
syntax parser and variable length decoder circuit for 
inverse quantizing only a limited preselected 
number of DCT coefficients per macroWock arnJ for 
outputting zeros corresporxjing to the DCT coeffi- 20 
cients of the macroWock which are not inverse quan- 
tized, wherein the limited number of DCT 
coefficients per macroblock is less than a maximum 
possible number of DCT coefficients per macrob- 
lock. 25 

5. The decoder circuit of claim 4. wherein the limited 
preselected number of DCT coefficients per mac- 
roblock is less than 1 1 and wherein the maximum 
possible number of DCT coefficients per macroblock 30 
is 64. 

6. The decoder circuit of claim 4, further conrtprising an 
inverse DCT circuit coupled to the inverse quantizer 
circuit for performir>g an inverse DCT operation on 35 
the limited preselected number of DCT coefficients 
per macroblock. 

7. The decoder circuit of claim 1 . further comprising: 

a syntax parser and variable length decoder 40 
circuit coupled to the channel buffer; 

a downsannpler coupled to the syntax parser 
and variable length decoder circuit for downsam- 
pling the video pictures output by the syntax parser 
to thereby reduce the amount of resolution of each 45 
picture and the anx)urTt of video data used to repre- 
sent each video picture. 

8. The decoder circuit of claim 7, wherein the pictures 
further include motion vectors, and wherein the so 



decoder circuit further comprises: 

a frame buffer coupled to the downsampler 
for storing the reduced resolution pictures generated 
by the downsampler; and 

a motion compensation circuit coupled lo the 
frame buffer, the motion compensation circuit includ- 
ing: 



24 

i. an upsampler circuit for upsampling the 
reduced resolution pictures to generate predic- 
tion reference pictures: 

ii. an interpolation circuit for interpolating the 
predication reference pictures; 

iii. a motion compensated prediction module for 
applying the motion vectors to the prediction ref- 
erence pictures to generate new pictures; and 

iv. a downsampler circuit coupled to the motion 
compensated prediction module for downsam- 
pling the new pictures to the same resolution as 
the pictures stored in the frame buffer. 

9. A method of decoding variable length encoded dig- 
ital video data including pictures, to a resolution less 
than the full resolution of the pictures, each picture 
including a plurality of macroblocks, each macrob- 
lock including a plurality of DCT coeffkaents, the 
method connprising the steps of: 

receiving digital vkjeo data; 
parsing the digital video data to identify niac- 
roblocks included in the digital video data arxf the 
DCT coefficients included in the identified macrob- 
locks; 

discarding from the digital video data the DCT 
coefficients included in each macroblock that 
exceed a preselected number of DCT coefficients; 
and 

storing the digital video data in a coded data 

buffer. 

1 0. The decoding method of claim 9, wherein each DCT 
coefficient is represented by a number of bits ar»d 
wherein each macroblock is represented by the total 
number of bits representing the DCT coefficients 
included in the macrot^locK the method further com- 
prising the step of: 

prior to storing the digital video data in a 
coded data buffer, discarding DCT coefficients from 
each identified macroblock until each identified mac- 
roblock is represented by a total number of bits that 
is less than a preselected maximum number of bits. 

1 1 . The method of claim 10. further comprising the steps 
of: 

monitoring the amount of data stored in the 
coded data buffer to determine coded data buffer 
occupancy; and 

dynamicly adjusting the total number of bits 
allowed per macroblock from the preselected maxi- 
nrtum number to a smaller number as coded data 
buffer occupancy increases to insure that the coded 
data buffer does not overflow. 

55 1 2. The method of claim 9. further comprising the steps 
of: 

- syntax parsing and variable length decoding 
the video data stored in the coded data buffer to gen- 
erate a variable length decoded video data stream 
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including macroblocks, each macroblock including a 
plurality o1 DCT coefficients; and 

inverse quantizing a preselected number of 
DCT coefficients included in each nriacroWock of the 
variable length decoded video stream and output- 5 
ting zero values corresponding to the macroblock 
DCT coefficients which are not inverse quantized. 

13- The method of claim 12. further comprising the step 
of: 70 

performing an inverse disaete cosine trans- 
form operation on the inverse quantized video data. 

1 4. The method of claim 1 3. further conrprising the steps 

of: 15 
downsampling the video data to reduce the 

resolution of the inverse quantized video data and 

to reduce the amount of memory required to store 

the inverse quantized video data; and 

storing the video data in a frame buffer. 20 

1 5. The method of claim 1 4, further comprising the steps 
of: 

upsampling the video data stored In the frame 
buffer; 25 

interpolating the upsampled video data; per- 
forming motion compensated prediction on the 
upsampled video data; and 

downsampling the upsampled video data 
upon which motion compensated prediction v^s so 
performed. 

16. The method of claim 15, wherein the downsampling 
operation performed on the video data after perform- 
ing motion compensated prediction is an exact 35 
inverse of the upsampling performed prior to per- 
forming motion compensated prediction. 

17. A method of decoding digital data representing a 
video frame, the digital data representing each video 
frame including video data representing a plurality 
of pixels, the video data representing each pixel 
including a set of bits, the method comprising the 
step of: 

truncating the least significant bit from each 
set of bits representing one of the individual pixels; 
and 

storing the truncated set of bits representing 
each one of the individual pixels in a frame buffer. 

18. The method of claim 1 7, further comprising the steps 
of: 

reading data out of the frame buffer; and 
appending a zero to each set of bits repre- 
senting one of the individual pixels as it is read out 
of the frame buffer. 

19. The method of claim 18, further comprising the step 
of: 



low pass filtering the data read out of the 
frame buffer. 

20. A digital picture -in -picture decoder circuit for decod- 
ing a primary picture to be displayed in a primary 
picture area and a second picture to be displayed in 
a secondary picture area, the secondary picture 
area being smaller than the primary picture area, the 
decoder circuit comprising: 

a full resolution video decoder for receiving 
and decoding a first set of digital video data including 
the primary picture; 

a first reduced resolution video decoder for 
receiving a second set of digital video data including 
the second picture and for decoding the received 
video data to generate a reduced resolution version 
of the second picture; and 

a post processor circuit coupled to the full res- 
olution decoder circuit and the reduced resolution 
decoder circuit, the post processor circuit operating 
to combine the primary full resolution picture and the 
reduced resolution version of the second picture into 
a single picture. 

21. The digital picture-in-picture decoder circuit of claim 

20, wherein the full resolution video decoder is a 
HDTV decoder and wherein the reduced resolution 
videodecoder is a reduced resolution videodecoder 
that generates pictures 1/4 x 1/4 the size of the full 
resolution video decoder. 

22. The digital picture-in-picture decoder circuit of claim 

21, further comprising a second reduced resolution 
video decoder coupled to the postprocessing circuit 
for decoding a reduced resolution version of a third 
picture; and 

wherein the post processor circuit operates 
to combine the reduced resolution version of the 
third picture with the first full resolution picture and 
the reduced resolution version of the second picture 
to form the single picture. 

23. The digital picture-in-picture decoder circuit of claim 
20, wherein the second set of digital video data 
includes a plurality of macroblocks. each macrob- 
lock including a plurality of DCT coefficients; and 

vkrtierein the first reduced resolution video 
decoder comprises: 

preparser means for receiving the second set 
of digital video data and for renrroving from each 
macroblock DCT coefficients that exceed a prese- 
lected maximum number of DCT coefficients per 
macroblock; and 

a channel buffer coupled to the preparser 
means for temporarily storing the video data output 
by the preparser means. 

24. The digital picture-in-picture decoder circuit of claim 
23. wherein the first reduced resolution video 
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decoder further comprises: 

a syntax parser and variable length decoder 
circuit coupled to the channel buffer for variable 
length decoding the second set of digital video data; 

a downsampler coupled to the syntax parser 5 
and variable length decoder circuit for downsam- 
pling the variable length decoded video data to 
thereby reduce the amount of resolution of the sec- 
ond picture and the amount of video data used to 
represent second picture. w 

25. A method of decoding variable length encoded dig- 
Kat video data including slice stuffing data, start code 
zero stuffing data, and macroblocks. the nriacrob- 
locks representing pictures, to a resolution less than is 
the full resolution of the pictures, the method com- 
prising the steps of: 

receiving the digital video data; 

parsing the digital video data to identify bit 
slice stuffing and start code zero stuffing data; 20 

discarding the bit slice stuffing data and start 
code zero stuffing data; and 

storing the remaining video data in a channel 

buffer. 

25 

26. The method of claim 25, further conprising the steps 
of: 

variable length decoding the video data 
stored in the channel buffer to generated fixed length 
video data; and 30 

downsampling the f ixed length video data. 

27. A method of decoding digital video data including an 
interlaced picture, to a resolution less than the full 
resolution of the interlaced picture, the interlaced 35 
picture including a plurality of maaoblocks erKoded 

in accordance with one of a field OCT basis or a 
frame DCT basis, the method comprising the steps 
of: 

downsampling the macroblocks encoded on 40 
a field DCT basis on a field basis; 

downsampling the macroblocks encoded on 
a frame DCT basis on a frame basis: 

storing the downsampled macroblocks in a 
frame buffer. 45 

28. The method of claim 27, wherein the video data fur- 
ther comprises a non-interlaced picture, the method 
comprising the steps of: 

downsampling the non-interlaced picture on 50 
a frame basis. 
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FIG.3 (A) 
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FIG.3 (B) 
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FIG.3 (C) 
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FIG.3 (D) 
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FIG.3 (E) 
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CLAIMS INCURRING FEES 



The pre&ent European patent application compneed at the time of filing more than ten ctaim&. 

□ Only part of the claim& have been paid within the prescribed time limit The present European search 
report has been drawn up for the first ten claims and for those claims for which claims tees have 
been paid, namely ctaim(s): 



□ No claims fees have been paid within the prescrit>ed time limit. The present European search report has 
been drawn up for the first ten claims. 



LACK OF UNITY OF INVENTION 



The Search Division considers that the present European patent application does rK3t comply with the 
requirements of unity of invention and relates to several inventions or groups of inventions, namely: 



SEE SHEET B 
(in case of Lack of Unity) 



□ 



□ 



All further search fees have been paid within the fixed time limit The present European search report has 
been drawn up for alt claims. 



Only part of the further search fees have been paid within the fixed time limit. The presem European 
search report has been drawn up for those parts of the European patent application which relate to the 
inventions in respect of which search fees have been paid, namely claims: 



□ None of the further search fees have been paid within the fixed time limit. The present European search 
report has been drawn up for those parts of the European patent application which relate to the invention 
first mentioned in the claims, namely claims: 
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EP 95 11 5965 



Application Number 



The Search Division considers that the present European patent application does not comply with the 
requirements of unity ot invention and relates to several inventions or groups of inventions, namely: 

1. Claims: 1-16,25-26 

instant reduction of bit rate at reception of compressed 
data 



2. Claims: 17-19 

video decoding method with truncation of pixel bit sets 
before storage in video RAM 



3. Claims: 20-24 

a Picture-In-Picture decoder circuit containing full 
resolution decoder for main picture and reduced resolution 
decoder for inset 



4. Claims: 25-26 

removal of stuffing data before parsing 



5. Claims: 27-28 

video decoding method with reduction of the amount of data 
dependent on frame/field dct coding 
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